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I. INTRODUCTION 

The kinetic theory, introduced and developed in the 
second half of the nineteenth century by Clausius [l|, 
Maxwell 0, and Boltzmann ii aimed to derive the 
macroscopic properties of matter, which include the laws 
of thermodynamics, from the underlying microscopic 
movement, governed by the laws of mechanics. In prin¬ 
ciple, this task can be achieved if we assume that the 
macroscopic laws are connected to the microscopic laws. 
Whether the connection exists or not we cannot known 
a priori. But, considering that it is an experimental fact 
that the laws of mechanics, classical or quantum, are 
obeyed at the microscopic level by the same particles 
that constitute a macroscopic body, which obeys macro¬ 
scopic laws, it is reasonable to assume that the connec¬ 
tion exists. Once we assume this connection, the next 
task is to perform the actual derivation of macroscopic 
laws from the microscopic laws of mechanics. This task 
was in fact undertaken by the founders of the kinetic 
theory and many macroscopic laws were in fact derived. 
This includes the theorem of equipartition of energy, the 
Maxwell distribution of velocities 0, the Boltzmann H- 
theorem Q and the Gibbs probability distribution Q. 
However, many results cannot be said to have been de¬ 
rived from pure mechanics alone 0 ■ A new ingredient 
was introduced in the course of derivation, namely, the 
stochastic behavior, in most cases in an implicit form. 

The derivation from pure mechanics of the results just 
mentioned would be accomplished if we could show that 
the new ingredient, the stochastic behavior, is a conse¬ 
quence of the microscopic mechanical motion, which is 
deterministic. At first sight the random behavior seems 
to be in contradiction with a deterministic motion. How¬ 
ever, the results coming from the theory of deterministic 
chaos 0 has proven that a deterministic motion can be¬ 
have stochastically. In fact, the possibility of mapping 
chaotic dynamics into a stochastic process has already 
been addressed @. The Gibbs probability distribution. 


for instance, is believed to come from the underlying me¬ 
chanics through a stochastic behavior, although there is 
no known general derivation from pure mechanics. In 
some cases, the derivation is know [l^. In other cases, 
such as a system of hard spheres, numerical simulations 
of the equations of motion may, for instance, show the 
validity of the equipartition of e nerg y or may provide the 
macroscopic properties directly |lll |. 

The reasoning and examples given above lead us to 
presume that the macroscopic properties are obtained 
from microscopic mechanics in two major steps: (I) from 
the underlying mechanics to a probabilistic or stochas¬ 
tic approach and (2) from this approach to the macro¬ 
scopic properties. This is particularly clear in the case 
of equilibrium thermodynamic properties, which are de¬ 
rived from the Gibbs probability distribution, which in 
turn comes from the underlying mechanics, a step not 
yet fully demonstrated and known as ergodic hypothe¬ 
sis. The first step will not concern us here. The second 
step, which is the purpose of the present paper, aims to 
derive the macroscopic properties, which include equi¬ 
librium and nonequilibrium thermodynamic properties, 
from a stochastic approach. 

The stochastic approach to equilibrium and nonequi¬ 
librium thermodynamics or, in short, stochastic thermo¬ 
dynamics, which is the second step of our scheme and the 
subject of the present paper, has been adopted by several 
authors and become a consistent theory of nonequilib¬ 
rium thermodynamics . An important step in this 

direction occurred when Schnakenberg El introduced 
the stochastic definition of entropy production rate which 
has a fundamental role in our approach in addition to 
the probabilistic definition of entropy itself, introduced 
by Boltzmann Q and generalized by Gibbs 

Our approach here is based on the adoption of a 
Markovian stochastic evolution on a discrete or on a con¬ 
tinuous space and on two assumptions concerning en¬ 
tropy. The first is the definition of entropy itself and the 
second, the definition of entropy production rate. Based 


2 


on these assumptions we will consider systems in equilib¬ 
rium and out of thermodynamic equilibrium and how the 
macroscopic nonequilibrium laws can be derived from the 
stochastic dynamics, which is the second step mentioned 
above. We will treat some fundamental issues that have 
barely been considered or that have not been addressed 
in the context of stochastic thermodynamics. This in¬ 
cludes several thermodynamic results of s yste ms in equi¬ 
librium (Mill and out of equilibrium [s^SSj j such as the 
quasi-equilibrium processes, the convexity of the equi¬ 
librium surface in thermodynamic space, the monotonic 
time behavior of thermodynamic potentials, including en¬ 
tropy, the bilinear form of the entropy production rate, 
the Onsager coefficients and reciprocal relations, and the 
nonequilibrium steady states of chemical reactions. 


The stochastic approach in continuous state space was 
used by Einstein [^, Smoluchowski and Langevin 
to explain the Brownian motion. It was generalized 
to the case of Brownian particles subject to an external 
force by Fokker , Smoluchowski , Planck , and 
Ornstein and the equation governing the time evo¬ 
lution of the probability distribution became known as 
the Fokker-Planck equation. Kramers extended the 
Fokker-Planck equation to the case of a massive parti¬ 
cle and studied the escape of a Brownian particle over a 
potential barrier arriving at the Arrhenius factor. 


Markovian stochastic dynamics |67H70| has been used 
in various problems in physics, chemistry and biology, ei¬ 
ther in continuous or discrete state space. In the former 
case, the evolution of the probability distribution is gov¬ 
erned by a Fokker-Planck equation and in the later by 

a master equation. _ We mention the study of chemical 

1^, population dynamics and 
. ^. 1 , and biological systems in general 
- IT^ . We wish to mention particularly 
the stochastic models with many degrees of freedom such 
as the so called stochastic lattice models usually used 
to describe phase transitions and criticality in physics, 
chemistry and biology 



evolution equation, in this case a master equation, 

w - (1) 
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where Wij denotes the transition rate from state j to 
state i. In this section and the next we will consider 
transitions with the following property: if the rate Wij 
of the transition j ^ i is nonzero then the rate Wji of the 
reverse transition i j is also nonzero. Later on, in the 
study of the Fokker-Planck, we will have the opportunity 
to treat the case in which the reverse transition rate may 
vanish. 

As mentioned above, the derivation of the macroscopic 
properties, including the laws of thermodynamics, is car¬ 
ried out by the introduction of two assumptions concern¬ 
ing entropy. The first is the definition of entropy itself. 
The entropy S' of a system in equilibrium or out of equi¬ 
librium is taken to be the expression 

= ( 2 ) 

i 

which is the extension of the equilibrium Boltzmann- 
Gibbs entropy to nonequilibrium situations, where ks 
is the Boltzmann constant. 

The second assumption concerns the definition of the 
production of entropy. This quantity should meet two 
fundamental properties. It must be nonnegative and van¬ 
ish identically in thermodynamic equilibrium. Following 
Schnakenberg we assume the following expression 
for the entropy production rate 

nw = ^In 

ij J * ^ V / 

which is clearly nonnegative because each term is of 
the form (x — y)ln(x/y). This form of entropy produc¬ 
tion rate has been used by several authors ullii,!!!- 
1^ . 1^ . 113, m, no, m, llOl within stochastic dynamics and 
applications. 


II. MASTER EQUATION 
A. Entropy and entropy prodnction 

We assume that the system follows a microscopic 
stochastic dynamics. More precisely, we assume that 
the system is described by a continuous time Markovian 
stochastic process. Considering a discrete space of states, 
this assumption amounts to say that the time evolution 
equation is set up once the transition rates are given. 
The transition rates play thus a fundamental role in the 
present approach and we may say that a system is con¬ 
sidered to be theoretically defined when this quantity is 
given a priori. Given the transition rates, the probabil¬ 
ity Pi{t) of state i at time t is obtained by solving the 


B. Entropy flux 

Let us consider the time variation of the average of a 
state function such as energy, given by 

U{t) = Y,E,P,{t). (4) 

i 

Using the master equation ([T]) it follows that 
dU 

m = *- 

where 

‘^>u{t)=Y,{E,-E,)W,,P,{t), (6) 
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is the total flux of energy from outside to the system. 
Equation ([S]) represents the conservation of energy. 

Equation of the type m is valid for any conserved 
quantity and this is not the case of entropy. For instance, 
in a nonequilibrium stationary state the total flux of en¬ 
ergy vanishes but not the total flux of entropy, which is 
nonzero because entropy is continuously being produced. 
The equation for the time variation of entropy S should 
be written as 


dt 


= n - $, 


(7) 


where $ is the flux of entropy from the system to the 
outside and If is the entropy production per unit time, 
given by equation ([3]). It is common to write diS/dt and 
deS/dt for the entropy production rate and entropy flux, 
respectively, but we avoid this terminology because these 
quantities are not in fact time derivatives of any quantity. 

Taking the time derivative of equation m and using 
the master equation o, we may write the time derivative 
of entropy as 


-^ = ^B W - In-Pz W, (8) 

-ij 


or, in an equivalent form, 

f(9) 

ij J \ ' 

Comparing with ([7]) we see that the right hand side of 
this equation should equal 11 — Using the definition of 
n, given by ([3]), which we write in the form 


which is the Clausius inequality [s^- In equilibrium, 
AS* = J dQ/T, equality that was used by Clausius to de¬ 
fine entropy. The difference between AS' and the integral 
J dQ/T, which is the production of entropy, represents 
accor ding to Clausius the “uncompensated transforma¬ 
tion” [s^- 

It is common in the recent literature to use another 
nomenclature for the entropy production 11, the entropy 
flux $ and the time derivative of entropy dS/dt. The 
quantities that correspond to the time integral of these 
three quantities are called, respectively, the total entropy 
change, the environment entropy change and internal en¬ 
tropy change [3, [ill . 


C. Thermodynamic equilibrium 

The microscopic definition of thermodynamic equilib¬ 
rium, from the static point of view, is usually charac¬ 
terized in terms of the Gibbs probability distribution. 
From the dynamic point of view, the description of equi¬ 
librium by the Gibbs distribution is necessary but not 
sufficient. There are examples [8§ - l3| of spin models 
that are described by the Gibbs distribution but are not 
in thermodynamic equilibrium in the sense that entropy 
is continuously being generated. From a dynamic point 
of view, the thermodynamic equilibrium is characterized 
by the vanishing of the entropy production rate and, of 
course, by a time independent probability distribution. 
The vanishing of ([3]) gives 

W^jPj = (15) 


n(t) 


In 

ij 


W,,Pj{t) 

Wj,p,{ty 


( 10 ) 


and comparing with equation ([9]) we get the flux of en¬ 
tropy from the system to outside 


$(t) = kBYZw^jPj{t)\n 

ij 


Ui' 


( 11 ) 


which is equivalent to 


which is the detailed balance condition, that character¬ 
izes the thermodynamic equilibrium [14| . and equivalent 
to microscopic reversibility. 

In the stationary state, that is, when the probability Pi 
is independent of time, the right hand side of o vanishes, 
that is. 


- W,.py = 0, (16) 

j 


- W(,iP,(i)}ln^. (12) 

ij 

The integration of ([7]) in a time interval will lead us 
to the Clausius inequality. Indeed, from equation © we 
may write 

AS = judt- J ^dt. (13) 

If we identify the entropy flux $ as the ratio between the 
heat flux dQ/dt and the temperature T of the environ¬ 
ment, then f ^dt = — f (dQ/T). But the first integral is 
nonnegative because 11 > 0 so that 

(14) 


which we may call global balance. The reversibility con¬ 
dition (USD can thus be understood as detailed balance 
condition because each term of the global balance equa¬ 
tion vanishes. Although the global balance is a necessary 
condition for reversibility it is not a sufficient condition. 

Considering that the equilibrium distribution P/ is 
known, the solution of (USD for the transition rate is 

, (17) 

where Kij is symmetric, that is, Kij = Kji. The transi¬ 
tion rates for the various situation in which the system 
is found in equilibrium in the stationary state can now 
be constructed. For an isolated system (microcanonical 
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ensemble) the equilibrium probability distribution Pi is a 
constant whenever the energy function Ei equals a given 
energy, say U, and vanishes otherwise. Therefore, in this 
case Wij = Kij when Ei = Ej and vanishes otherwise. 
In short, Wij = Wji. 

For a system in contact with a heat reservoir (canonical 
ensemble) at temperature T, the equilibrium probability 
distribution is given by 

= (18) 


where (3 = 1/fcsT, so that, in this case the transition 
rate fulfils the relation 


= p-fliEi-Ej) 


and is given by 

Wij = Kije-f^^^'-^^P'^. 


(19) 


( 20 ) 


That is, the entropy of an isolated system is a monoton- 
ically increasing function of time. 

Next we consider the canonical distribution which de¬ 
scribes the contact of a system with a heat reservoir. The 
transition rate is given by dsni), which replaced in the en¬ 
tropy flux dD gives 

$ = -kBPY.^W,jPj{t) - WMt)]E,. (26) 


Using the master equation o, the flux of entropy can be 
written in the form 


1 dU 
T~dt’ 


(27) 


where U is the average of energy, given by ®. Equa¬ 
tion (Ell) shows that the quantity $ is proportional to 
dU/dt. Notice that (l27l) implies that $ vanishes in the 
equilibrium regime (t —oo) as it should. 

Equation © gives 


If, in addition to be in contact with a heat reservoir, 
the system is in contact with a reservoir of particles, then 


dt dt 


(28) 


pe ^ ( 21 ) 

where /i is the chemical potential and is the number 
of particles. In this case the transition rate fulfils the 
relation 


If we define the free energy by F = U — TS and take into 
account that T is constant, that is, does not depend on 
time, we get 


dt 


= -Til, 


(29) 


di = p-PiEi-Ei)+l3p,ini-nj) 

Wp 


and is given by 

. _ p-_^-^{Ei-Ej)/2+l3ii(ni-nj)/2 
'^3 ^ 3 ^ ’ 


where again = Kji. 


D. The approach to equilibrium 


( 22 ) 


so that 


^< 0 - 

dt 


(30) 


That is, the free energy of a system in contact with a heat 
reservoir is a monotonically decreasing function of time. 
In other terms, the free energy decreases monotonically 
to its equilibrium value. 

Equation ([301) is also the expression of the Boltzmann 
H-theorem Q. Indeed, the Boltzmann H function is de¬ 
fined by 


Let us consider the transient regime of a system that 
approaches equilibrium. The time dependent probability 
distribution is the solution of the master equation 
with transition rates that satisfy the detailed balance and 
is appropriate for each type of contact of the system with 
the environment. 

We treat first the case of microcanonical distribution, 
which describes an isolated system. In this case, as we 
have seen, Wij = Wji so that the entropy flux m van¬ 
ishes identically, $ = 0. Therefore, 


so that 


H{t) = J2P,{t)\n^, (31) 

i i 

where P/ is the equilibrium canonical distribution given 
by equation ((T5)) . It is straightforward to show that P = 
Eq + H/P, where Fq does not depend on time. Therefore, 
the inequality (15(11) is equivalent to dH/dt < 0, which is 
the Boltzmann H-theorem. 

The grand canonical distribution describes the contact 
of the system with a particle reservoir and with a heat 
reservoir. The transition rate for this case is given by 
(1231) , which replaced in the expression m and using the 
master equation allows to reach the following expres¬ 
sion for the entropy flux 


dS 

— > 

dt 


0 . 


(25) 


1 dU fi dN 
T~dt ^ T~dt’ 


(32) 
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where U is the average energy, given by 0 , and N is the 
average number of particles, 

N[t) = Y,n^P^(t). (33) 

i 

Taking into account that dS'/dt = 11 — we get 



^_T— 
dt dt 


dN 


which can be written as 


dtp 

dt 


-Tn, 


(34) 


(35) 


where 4> = U — TS — fiN is the grand thermodynamic 
potential and we have taken into account that T and /r 
are constant. Since 11 > 0 it follows that dp/dt < 0. 

Let us integrate equation p4l) from an initial time t = 
to to infinity, 


poo 

iU-Uo)-T{S-So)-t^{N-No) = -T Udt, (36) 

Jto 


from which follows the inequality 


{U-Uo)-T{S-So)-ti{N-No)<0, (37) 

because 11 > 0. Taking into account that, for large 
enough times, the system reaches equilibrium at a tem¬ 
perature T and imposing that at t = to the system was in 
equilibrium, at a different temperature, say Tq, we may 
conclude from the inequality ([37|l that U, S and N make 
up a convex surface, in accordance with equilibrium ther¬ 
modynamics. 


E. Quasi-equilibrium 

It is common to state the laws of equilibrium ther¬ 
modynamics in terms of thermodynamic processes. This 
seems at first sight contradictory because a process im¬ 
plies a change in the thermodynamic state and thus a 
displacement from equilibrium. To overcome this prob¬ 
lem, one introduces the quasi-static process, a process 
which is so slow that the system may be considered to be 
in equilibrium. We will show below that the production 
of entropy in this process is negligible so that in fact the 
system may be considered to be in equilibrium. In which 
sense the production is negligible will be seen below. 

Let us consider a system in contact with a heat bath 
and a particle reservoir whose temperature and chemical 
potential, understood as control parameters, are slowly 
varying in time. To describe this situation we assume a 
time dependent transition rate Wij{t) of the form ( 1 ^ . 
where Kij(t) may depend o time, that is, 




FIG. 1: A path in the T, p space and the corresponding tra¬ 
jectory in the thermodynamic space S, U, N. If the variations 
in T and are very slow, then the trajectory in the thermo¬ 
dynamic space approaches and remains on a certain surface 
which has the property of convexity and is identified as the 
thermodynamic equilibrium surface. The portions of the tra¬ 
jectory outside and on the surface are represented by dashed 
and solid lines, respectively. 


where Kij{t) = Kji{t), so that 


ITy(t) 


(39) 


where /3 = I/fcsT and T{t) depends on time and the 
chemical potential /r(t) also depends on time. We assume 
moreover that dp/dt = a and d^/dt = 7 are small and 
are both of the same order of magnitude. 

Replacing equation (l3^ in expression (ED for the en¬ 
tropy flux and after some straightforward algebraic ma¬ 
nipulation we reach again the result 


IdU ndN 
tHI ^ T~dt' 


Now dS/dt = n — $ so that 


dt dt 



-Til. 


(40) 

(41) 


Let us now find the solution of the master equation. 
To this end, we write the solution as 


P,{t)=P*{t)+Mt), (42) 


where 


P*it) = exp{-P{E, - urii)}, (43) 

and S(,9, /i) is a time dependent quantity such that P*(t) 
is normalized at any time, and is small when compared 
to P*. It is important to bear in mind that although 
P* (t) obeys the relation 


W,,{t)P*{t) = W,Pt)P*{t), (44) 

and can be interpreted as a probability distribution, it 
is not the solution of the master equation, given by O- 
The substitution of P* on the master equation o makes 
the right hand side equal to zero but not the left hand 
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side. Replacing equation (l42l) into the master equation, 
we get, up to first order in the perturbation A^, 

jP:(t) = E , (45) 

3 

Now 


dP* dP* dP* 

~dr ~ 


(46) 


which, in view of equation (|45|) . implies that the per¬ 
turbation Aj{t) is of the order of a and 7 . From the 
expression (HDll for the entropy flux 4), it follows that 4> 
is also of the order a and 7 . On the other hand, if we 
consider the expression ([3]) for the entropy production 
n, it follows that 11 is of of second order in a and 7 . 
Therefore, in the quasi-equilibrium regime, in which we 
consider only terms up to first order in a and 7 , the re¬ 
lation dS/dt = 11 — 4) becomes dS/dt = —4), that is, the 
production of entropy vanishes when compared with the 
flux of entropy. Using this result it follows from (|4T]) that 
the following thermodynamic relation holds 


dt dt ^ dt 


(47) 


Let us take a look at the thermodynamic space spanned 
by the variables S, U and N. From the solution of the 
master equation, we may determine these quantities as 
a function of time by using the definitions ([ 2 ]), (jH) and 
HMD- The evolution of the system may be represented by 
a trajectory of a point in this space, as shown in figure 
[TJ The representative point will describe a generic tra¬ 
jectory in this space. But, if T and ^ start to vary very 
slowly, the trajectory, according to the result (HTl) . will 
approach and remain on a certain surface of this space, 
as seen in figure [H According to (l47l) , the surface is rep¬ 
resented by the equation 


dU = TdS -b fidN, (48) 


so that the temperature T of the thermal reservoir be¬ 
comes identified as the tangent to the surface U{S,N) 
along the S direction, T = dU/dS, and can thus be in¬ 
terpreted as the temperature of the system. Similarly, 
the chemical potential of the particle reservoir becomes 
identified as the tangent to the surface U{S, N) along the 
N direction, = dU/dN, and can thus be interpreted as 
the chemical potential of the system. Notice that, accord¬ 
ing to the inequality (IWl) . this surface has the property 
of convexity. 

We should remark that far from equilibrium, the tem¬ 
perature of the system cannot be defined because S, U 
and N are not connected by relation (H51) . The same can 
be said about the free energy of systems far from equi¬ 
librium. Notice however that the quantity F = U — TS, 
defined previously and called free energy, is not properly 
a property of the system because T is the temperature of 
the reservoir and not the temperature of the system, since 


it cannot be defined. In equilibrium or quasi-equilibrium, 
however, it becomes a well defined quantity as much as 
the temperature. It is worth mentioning in addition that 
according to the approach just presented, the control pa¬ 
rameters should be the thermodynamic variables known 
as thermodynamic field variables 51|. 


F. Fluxes and forces 

We consider here the contact of a system with two dis¬ 
tinct reservoirs. To treat this situation properly, we as¬ 
sume that each pair of states {i,j) is either associated to 
the first reservoir or to the second reservoir or to neither 
of them. In other words, the set of pairs (i-,j) is parti¬ 
tioned into three subsets, associated to the first reservoir, 
to the second reservoir and neither of them, which we de¬ 
noted by A, B and C, respectively. The transition rates 
associated to the reservoirs 1 and 2 are denoted by Wl 
and Wfj , respectively, and are assumed to be of the same 
form of (1^ . that is, 

_ J^r ^-Pr{Ei-Ej)/2+Pr-tJ.T(rii-nj)/2 ^ 

for r = 1,2, where is symmetric as before. In addi¬ 
tion, Kfj depends on Ti and p,i and is nonzero only if 
(i,j) € A, and Kfj depends on T 2 and fi 2 and is nonzero 
only if {i,j) G B. We are denoting by Ti and and T 2 
and fi 2 the temperatures and chemical potentials of the 
two reservoirs, and jdr = l/ksTr- The full transition rate 
is given by 

IU„-=IF°+11^1 +11^2, (50) 

where IU° may be nonzero only if {i,j) G C. In this 
case it is nonzero if Ei = Ej and = Nj, in which case 

W° = W°-. 

’-o 3 I' , 

In the following, we consider the stationary regime for 
which the stationary probability distribution Pi fulfils the 
global balance m, but not the detailed balance. In the 
present case 11 = 4) and using the expression CB) we may 
write 

n = E (51) 

r=l,2 ij A 

where the first summation runs only over r = 1,2. The 
terms corresponding to r = 0 vanish because IUj° = 0 
or because IU°- = Wj). Replacing expression in this 
equation, the entropy production can be written as 

n = E E - Ei) - fXrinj - n,)]. ( 52 ) 

r—1,2 ij 

Now, the flux of energy Ju and the flux of particles Jn 
from reservoir 1 into the system are given by 

Ju = Y.^l3P3{P—^3\ (53) 

ij 
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= (54) 

ij 

The substitution of (1551) and (155)1 into (1551) and the use of 
the global balance condition (IT6)) . allow us to write the 
entropy production rate in the bilinear form [H, [s^ 


If — XuCfu + XnJni (55) 

where and are the thermodynamic forces 


1 1 

-^U rj^ rji 7 

J^2 


- f?, (56) 

-tl -t 2 


conjugated to the flux of energy and particles, respec¬ 
tively. 


G. Onsager coefficients 

When T 2 = Ti and /i 2 = that is, when Xu = 0 and 
Xn = 0, the fluxes Ju and Jn vanish. Therefore up to 
linear terms in Xu and Xn we expect the linear behavior 
of the fluxes. 


Lun = \Yl - Ej), (64) 

'i-3 

Lnu = ^ X! (65) 

Lnn — 2 ^ijEj {bj — bi){rii — rij), (66) 
ij 

where we have used the detailed balance condition (1501) . 
In the form given by equations (1551) and (1551) we can¬ 
not tell whether the coefficients Lnu and L„„ are equal. 
Next we perform a transformation to find expressions 
that will show that these coefficients are indeed equal to 
each other. 

Replacing (1551) into (1T51) . and expanding the result up 
to linear terms in Xu and Xn, we end up with the fol¬ 
lowing equations for Oi and bi, 

^ - a,) + ^ ^ W* P/ {E, - P.) = 0, (67) 

3 3 


Lfu — LuuXu ~\~ LunXn, 


3n — Lnu Xu LnnXn- 


(57) 


(58) 


The coefficients Luu, T«ra, Lnu and are the Onsager 
coefficients. According to Onsager, the cross coefficients 
are equal, Lun = Lnu, which is the Onsager reciprocal 
relation. In the following we will derive expressions for 
these coefficients an prove the reciprocal relation. 

We will suppose that Ti and fii are fixed and let T 2 
Ti and ^2 ^ Li- Pi b® fbe probability distribution 
corresponding to the equilibrium case, given by 

p-e ^ (59) 

The transition rate Wj® obeys the detailed balance 

MA-p; = w^^Pt, (60) 

and is given by 

^re _ j^e ^-l3i{Ei-Ej)i2+l3iiJ.i{ni-nj)j2^ (g^) 

where Kfj = + K*^ + Ar°- and K*^ equals when 

T 2 Ti and fj ,2 Li and = IT°. 

The stationary solution Pi of the master equation (HD) 
is written as 


P = J^®(l + a,A„ + 6,A„), 


(62) 


up to linear term in Xu and A„. Replacing into the 
expressions (1551) and ((551) we get the Onsager coefficients 
in the form 

Euu = IY1 - Ej), (63) 


^ W^^P^ib, -b,) + —J2 w:^p^{n, - n.) = 0, (68) 


where W*^ equals when T 2 —>■ Ti and ^2 —f and 

is given by 

W*j = (gg) 

Multiplying (1571) by Ei and by ai and summing in i we 
are lead to two equations from which we may obtain the 
following expression for Luu, 

ij 

(70) 

Multiplying (I68p by rii and by bi and summing in i we 
are lead to two equations from which we may obtain the 
following expression for Lnn, 

ij ij 

(71) 

Multiplying (1571) by rii and (1551) by ai and summing in 
i, we get two equations from which we reach an expres¬ 
sion for Lnu- Similarly, multiplying (157)) by bi and ()68l) 
by Ei and summing in f, we get an expression for Lun 
which is equal to Lnu, proving the reciprocal relation. 
The expression for these two quantities is given by 

Lun = Lnu = ^ijPj (Ej — Ei){nj — Hi) 


-^Y.^^3P!ih-b^){a,-ai). 


(72) 


It is worth mentioning that in the course of derivation 
of these expressions, we have made use of the detailed 
balance condition, which is thus a necessary condition to 
prove the reciprocal relation. However, the expressions 
for the Onsager coefficients do not depend on the equilib¬ 
rium distribution alone but depend also on the deviations 
tti and bi- 


E EA EAE 

(p-@-® 

(D-®-® 

I lA lAB 


H. Several species of particles 

We will now treat the case of a system composed by 
several types of particles in contact with two particle 
reservoirs, denoted by 1 and 2. In the steady state, fluxes 
of particles of the various types will be established be¬ 
tween the two reservoirs. Each reservoir is in fact a set 
of reservoirs, one for each type of particles. As before, 
denoting by the energy of state i and by the num¬ 
ber of particles of species k in state i, the rate of the 
transition j ^ i associated to the reservoir r and species 
k is given by 

where is the chemical potential of species k associated 
to reservoir r, and is symmetric. The reservoirs are 
also thermal reservoirs with a common temperature T 
and P = l/ksT. A transition rate that is not associated 
to any reservoir is denoted by IE?- and is assumed to be 
of the form 


W° = (74) 

which describes the contact with a heat reservoir at tem¬ 
perature T, where is symmetric. 

At the stationary state, the entropy production rate 
equals the flux of entropy and is given by 


r= 0 ,l ,2 k ij 


W] 


rk 


w:. 


(75) 


which follows from the general expression ([Tl]) . The sub¬ 
stitution of (1751) and (1751) into this expression gives 

n = ^ E E E (76) 

r=l,2 k ij 


FIG. 2: Transition diagram for a model for active transport 
across a cell membrane. The circles represent the possible 
states of a channel and the bonds represent the possible tran¬ 
sitions. The possible states of the channel are: a) open to 
exterior and empty (E), or holding a molecule (EA), or hold¬ 
ing two molecules (EAB); b) open to interior and empty (I), 
or holding a molecule (lA), or holding two molecules (lAB). 


where 

Xk = ^{A-ki) (79) 

is the thermodynamic force associated to species k. 

For the case of two types of particles, it follows from 
expression (1781) that XiJi + X 2 J 2 > 0 because H > 0. 
If Ai < 0, it is possible to have 77i > 0, as long as 
X 2 J 2 > |A'i|Ji, so that the flux of particles of type 1 
will occur against the chemical potential gradient. This 
is, for instance, a mechanism for the active transport 
across a cell membrane. A simple model [tS ] of this type 
of transport is examined next. 

A cell membrane is assumed to have a certain number 
of channels through which two types of molecules may 
cross the membrane from the exterior to the interior of 
the cell. The channels function independent of each other 
so that it suffices to consider just one of them. A channel 
may be open to the exterior, understood as reservoir 1, 
or to the interior, understood as reservoir 2, and can be 
either empty or hold a molecule A or two molecules, one 
A and another B. The possible states and transitions are 
shown in figure [7] 

Denoting by Wij the rate of the transition j ^ i, then 
^31, Wi3, W53 and u>35 are associated to the reservoir 1, 
whereas W 42 , W 24 , W 64 and W 46 , associated to the reser¬ 
voir 2. In this simple model, the energies of the state are 
assumed to be the same so that, according to (1751) they 
hold the following relations 


where the terms involving the energy vanish. Taking into 
account that the flux of particles of type k, from the 
reservoir 1 to the system, is given by 

(77) 

ij 

and using the total balance equation (1151) , we may write 
again the entropy production rate in the bilinear form 

Il = J2XkJk, (78) 

k 




(80) 

Wi3 

W 35 



(81) 

W24 

W46 


where and fig are the chemical potentials of molecules 
A and B associated to reservoir k. The other rates are not 
related to the reservoirs and are symmetric, ^21 = W 12 , 
W43 = W34, and 1^65 = UI 56 . Assuming that chemical 
potentials are given, the model has seven independent 
transition rates. 
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At the stationary state, the probability distribution Pi 
obeys the global balance equation 

^{wijPj - WjiPi) = 0 , (82) 

3 

but do not obey the detailed balance condition, which 
means that WijPj — WjtPi ^ 0 in general. The fluxes 
JA and Jb of molecules A and B, respectively, from the 
exterior to the interior, are given by 


Ja = W 13 P 3 - W 31 P 1 , 

(83) 

Jb = W 33 P 3 — W 53 P 3 , 

(84) 


and are nonzero because detailed balance does not hold. 
The entropy production rate is 11 = XaJa + XbJb 
where Xa = - A)/^ and Xb = (mb - Pb)/^'- By 

an appropriate choice of the transition rates, it is thus 
possible to have a flux of particles B against its chemi¬ 
cal potential gradient [tJ, that is, it is possible to have 
JTb > 0 and Xb < 0, as long as the flux of particles A 
agrees with the gradient of its chemical potential, that 
is, XaJa > 0 . 


III. CHEMICAL REACTIONS 
A. Equilibrium 


where /3 = I/^bT. 

To set up the transition rate W^j describing the con¬ 
tact of the system with the fc-reservoir we use the detailed 
balance condition with respect to the probability distri¬ 
bution (l 86 )) . 


'^'^13 _ fA. 

W^. Pf ’ 

3 


(87) 


where i and j are states such that condition (1851) is ful¬ 
filled, so that 


W'^ 


( 88 ) 


which leads us to the following form 

ij 5 


(89) 


where is symmetric, that is, = Afj), and is pos¬ 
itive if condition (jM]) is fulfilled and vanishes otherwise. 
The total transition rate Wij due to the contact with all 
reservoirs is written as the sum 


( 90 ) 

k=l 


We will be concerned in this section with a system com¬ 
posed by q species of particles that react among them¬ 
selves according to r reactions. The system is in contact 
with a heat reservoir and may be closed to particles or 
may be open and exchange particles with the environ¬ 
ment. This last situation is carried out by placing the 
system with particle reservoirs. We will treat in the fol¬ 
lowing the more general open case. The results for the 
closed case will readily be obtained from the results of 
the open case by formally imposing the vanishing of the 
particle flux. 

The system is placed in contact with q particle reser¬ 
voirs, one for each type of particles. Each particle reser¬ 
voir is also a thermal reservoir. The fc-th reservoir ex¬ 
changes heat, at temperature T, and only particles of 
type k, at a chemical potential /i^. Notice that all reser¬ 
voirs are at the same temperature T. The number of 
particle of species k in state i is denoted by 71^. If the 
fc-th reservoir causes a change from state j to state i then 

rii ^ Uj and = Uj , k' ^ k, (85) 

because we are assuming that the fc-th reservoir causes 
a change in the number of particles of type k but causes 
no changes in the number of particles of the other types. 

When the system is in thermodynamic equilibrium 
with the reservoirs, the probability distribution describ¬ 
ing the system is the Gibbs distribution 

J>e ^ (86) 


Notice that at most one of the q terms on the right hand 
side can be nonzero. 

Let us consider now the occurrence of chemical reac¬ 
tions. The number of particles of each species will vary 
not only because of the contact with the reservoirs but 
also because of the reactions. We consider the occurrence 
of r reactions described by the chemical equations 

9 

Y,^keBk = 0, £=l,2,...,r, (91) 

where Bk denotes the chemical formula of species k and 
Vki are the stoichiometric coefficients, which are negative 
for the reactants and positive for the products of the 
reaction. If the £-th reaction causes a change from state 
j to state i then 

Hi -rij = vu or ~'^’j = ~’^ke- (92) 

To set up the transition rate Wfj describing the change 
caused by the Gth reaction we assumed that it obeys the 
Arrhenius equation [^, 

W^- 

J* 

The most general form of the transition rate is 


(94) 
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where the prefactor is symmetric, that is, and 

is positive or vanishes according to whether the condition 
(IM)l is fulfilled or not. The transition rate Wij due to all 
reactions is written as the sum 

r 

( 95 ) 

Notice that at most one of the r terms on the right hand 
side can be nonzero. 

The full transition rate Wij describing the r reactions 
as well as the contact with the q reservoirs is given by 

W,j=W,j+W,j. (96) 

Again, just one of the two terms on the right hand side 
can be nonzero. 

In equilibrium, detailed balance should be obeyed for 
each one of the transition rates on the right-hand side 
of (IMl) . We have seen that this is the case of the transi¬ 
tion rates W^j, related to the contact with each reservoir, 
when the probability distribution is that given by (l86l) . It 
suffices therefore to impose detailed balance to the tran¬ 
sition rate associated to each chemical reaction. To this 
end we compare the ratio (1931) with the ratio 

^3 

obtained from (1861) and valid when the first of the two 
conditions in dMl) is fulfilled. The condition of detailed 
balance is obeyed when the two ratios are equal to each 
other, that is, when 

E = 0, (98) 

k 

for each reaction i. The same conclusion is obtained if 
we use the second of the two conditions in (IMl) . Equa¬ 
tion (|98l) is the well known equilibrium condition that 
should be fulfilled when chemical reactions take place in 
a system [IS in. In the presence of chemical reactions 
and in equilibrium, the chemical potentials of the chem¬ 
ical species cannot be independent but are related by 
(1981) . In other words, the equilibrium occurs only when 
the chemical potentials of the particle reservoirs are 
tuned so that (1981) is fullfiled. Otherwise, the system will 
be out of equilibrium, as we shall see next. 

B. Nonequilibrium regime 

Let us now suppose that the condition (155)) is not 
obeyed. In this case the detailed balance condition is 
not fulfilled and the system cannot be in equilibrium. 
Each reaction is shifted either to the products or to the 
reactants. That is, for a given reaction, either the prod¬ 
ucts are being created and the reactants being annihi¬ 
lated (forward reaction) or the reactants are being cre¬ 
ated and the products being annihilated (backward reac¬ 
tion). In this nonequilibrium regime the time variation 


in the number of particles has two parts: one due to the 
flux of particles from the reservoirs and the other due the 
creation and annihilation caused by the reactions. 

Using the master equation ©, we see that the average 
number of particles Nk of type k, 

iVfc(t) = E^?^.(0, (99) 

i 

evolves as 

^ = ( 100 ) 

ij 

According to (IMl) . the transition rate Wij has two parts, 
one related to the reservoirs, which is Wij, and the other 
related to the chemical reactions, which is Wij, so that 
(I100|) can be written in the form 

^ = E W.jPjin': - n^) + <i>k, ( 101 ) 

where is given by 

^, = J 2 %P 3 (.n^-n';), ( 102 ) 

ij 

and describes the flux of particle from the fc-th reservoir 
to the system. The contact of the system with the fc-th 
reservoir, described by the transformation (|85|) . causes 
no changes in , k' ^ k. As a consequence, 

W)5 (nf-nf)=0, k^k', (103) 

Using (IMl) and the result (I103L the flux of particle (11021) 
is written as 

= (104) 

ij 

The summation in the right-hand side of (IIOII) de¬ 
scribes the change in the number of particle due to the 
chemical reactions. To describe properly this part, which 
corresponds to the creation and annihilation of particles 
caused by the reactions, it is convenient to use a new 
set of variables in the place of the set of variables 
k = l,2,...,q. The new variables are denoted by erf, 
£ = 1,2,... ,r and xf, £ = r + l,r + 2,... ,q and defined 
by the linear transformation 

r 

+ E = 1,2,...,(?, (105) 

e=i 

where the quantities x\, k = 1,2 ... ,r are not variables 
but arbitrary constants chosen to be the same for all i. 
If a transformation nf —>■ nf is performed according to 
the £-th chemical reaction (l9T]) . described by the trans¬ 
formation (1551) , the variables x\ remains unchanged, that 
is, Xj = x’i. As a consequence of this invariance, 

w!^ix’y-x’;) = o. 


(106) 
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In addition, according to the transformation (|M|) . the 
variables cr™, m £, associated to the other reactions 
remain unchanged, ct™ = cr'", and, as a consequence, 

rn^L (107) 

Using (|105l) and the results (llOOp and (I107p . we obtain 
^ WijP^j (nf -n])=Y^ VMXi, (108) 

ij t 

where the quantity Xi is 

= (109) 

ij 

The variation in the number of particles can then be writ¬ 
ten as 

=^VMXt + ‘^k- ( 110 ) 


When Xi > 0, the £-th chemical reaction is shifted to 
the right, in the direction of the products. When xe < 0, 
it is shifted to the leftt, in the direction of the reactants. 
The extent of reaction is defined as the average of af, 


= (Ill) 

i 


From the master equation and using the properties (11031) 
and (|107l) . we get 



( 112 ) 


and we may conclude that the quantity xi is the rate of 
the extent of the Uth reaction. 

The time variation of the internal energy is written as 


dt 




(113) 


Let us now consider the time variation of entropy 

— =n-$, (114) 

where the flux $ is given by ([3]) which, by the use of (1351) , 
(1351) and ((331) . is given by 


ij k 




ij e ji 


(115) 


+ (116) 

ij k 

Using equations pi3l) and (11041) we may write the flux 
of entropy as 

1 dU 1 , 

* = + (117) 


Substituting into (11141) and taking into account the equa¬ 
tions (1TT31) . we get 


dt 


= n 


1 dU 1 1 

T~dt ~ ^ 


Using (IllOl) . we reach the result 

dS I dU Ag I dNk 

~di ~ t’wT ~ E “ T E^*i' 


T dt 


dt 


(119) 


where Ag is the De Bonder affinity 


Ag = -Y^ 




( 120 ) 


associated to the £-th chemical reaction. 

In the stationary state dS/dt = 0, dU/dt = 0, and 
dNk/dt = 0, and we reach the following expression for 
the production of entropy in the stationary state [5^ Issl . 

[ 13 , [Hi 


n = E 

I 


Ag 

rjn X^1 


( 121 ) 


or, in the equivalent form, 

TT — 

~ ^ T dt' 


( 122 ) 


equation originally introduced by De Bonder [5|. In 
equilibrium there is no production of entropy and the 
affinities vanish, Ag = 0, in accordance with (1351) . and 
the rate in which the £-th reaction proceeds vanish as 
well, Xi = d^e/dt = 0. 

In a nonequilibrium stationary state, a flux of particles 
is continuously taking place, which sustain the chemical 
reactions. The quantities and Xi ^re nonzero, in gen¬ 
eral. At the same time there is a flux of heat toward the 
system, characterizing an endothermic reaction, or from 
the system, characterizing en exothermic reaction. To 
understand this situation we write down the variation in 
the energy, given by equation (11131) in the form 

^ = + (123) 


Substituting the rates (1551) and (1331) into this equation 
we get 


where Rg is the energy delivered by the £-th reaction per 
unit time, given by. 


^ = kBY.w,,p,i-pm-E,) 

^3 


u' 


( 124 ) 
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and $ii is the heat flux to the system, given by 

= (125) 

ij k 

In the stationary state dU/dt = 0 and <i>u = — J2e If 
< 0, the reactions are exothermic. If > 0, they 
are endothermic. Notice that there is no contribution to 
the entropy production rate coming from the flux of heat 
because the temperatures of the reservoirs are the same. 

Although we have considered a system in contact with 
one reservoir for each type of particles, the formulas can 
easily be adapted to the case in which the system is closed 
to some types of particles. If the system is closed to 
particles of type k, then it suffices to formally set fj,k = 0 
and W}j = 0 so that = 0 for this species. It is worth 
mentioning that in the case of a closed system, when 
there is no flux of particles from the environment, = 
0 for all species, and using equation (III8I1 . we see that 
dF/dt = —Til where F = U — TS is the free energy, so 
that dF/dt < 0. Therefore, the chemical reactions occur 
in a direction such that the variations in the number of 
particles will decrease the free energy [^ . 

As an example of the approach just developed we an¬ 
alyze a system with four species of particles and two re¬ 
actions, which are 

Bi + B2 = i? 3 , B3 = B2 + i? 4 , ( 126 ) 


We may now draw the following conclusion for the case of 
a nonequilibrium steady state situation, for which 11 > 0 . 
If /Ti > ^4 then xi > 0 and X 4 > 0 so that the two 
reaction equations are shifted to the right establishing 
a continuous annihilation of particles of type i?i, which 
come from reservoir Bi because $1 > 0 , and production 
of particles of type B 4 , which go to reservoir B 4 because 
$4 < 0 . 

C. Onsager coefficients 

In the nonequilibrium stationary state but close to 
equilibrium we may expand the rates of the extents of 
reaction xe in terms of the affinities Ag to get 

^ ( FgmAjYiy (131) 

m 

where Lgra are the Onsager coefficients. They obey the 
reciprocal relations, which we demonstrate next. 

We start by expanding the stationary probability dis¬ 
tribution Pi, that satisfies the global balance equation 
m, around the equilibrium distribution P/ given by 

pe = 1 -/3i5.-S/3 Efc ^ (132) 

Zj 

where the chemical potentials obey the equilibrium 
condition 


and represent the Michaelis-Menten mechanism in which 
a the substrate Bi is converted, in two steps, into the 
product B 4 by the action of an enzyme. The substrate 
Bi reacts with the enzyme B 2 giving rise to a complex B^ 
which in turn breaks up into the product B 4 and the en¬ 
zyme i? 2 - It is assumed that both reactions have reverses. 
The system is assumed to be closed to the particles B 2 
and i? 3 , and is in contact with reservoirs of particles of 
type Bi and B 4 . 

Using formula (11201) . and bearing in mind that we 
should set ^2 = 0 and /is = 0 in this formula, the affini¬ 
ties Ai and A 2 associated to the two reactions are given 
by 


Ai — /ii, A 2 — —/i4- (127) 

The variations in the number of particles of each species 
are 


dJVi 

dt 


-Xi + ^i> 


dt 


-X 1 +X 2 , (128) 


dNs 


— 77 ^ = X 2 + ‘I’4- (129) 


In the stationary state, Xi = X 2 = ‘I’l = —$ 4 , so that, 
using (II2II1 . the entropy production rate is found to be 


U=^{fi4-fi4). (130) 


E = 0- (133) 

k 

We assume an expansion of the form 

r q 

P^ = P/{l + J2R^gAg+ E a*fcA/ife), (134) 

1=1 k=r+l 

where A/ifc = /(^ — /(^. 

We also expand W//, given by (IMl) . around its value 
at equilibrium 

(135) 

to get 

Wt, = {1 + /3A/ife(nf - nf)/ 2 }. (136) 

The transition rate W/j needs no expansion because this 
quantity is also its value at equilibrium since it does not 
depend on the chemical potentials. 

Replacing the expansions (11341) and (11361) into the 
global balance equation dSl), we get 

r 

3 ^-1 

q 

+ E E (“■?■'= “ 0.ik)^d-k 

j k=r+l 
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+EE W*P^l3A^ik{n'i-n’;)=0, (137) 

j k 

where 


IV. FOKKER-PLANCK EQUATION 
A. Langevin equations 


=E%/- and W*=W:^+W,,. (138) 

k 

Using (110511 and taking into account relation (I133I1 . we 
see that 

E A/rfe(nf -n*) = 

k 

= E -E 

k=r+l 1=1 

which is replaced in (11371) to get an expression linear in 
Ag, and A/i^. Since the coefficients of Ag and A/i^ in this 
expression should vanish, we obtain 


In this section we are concerned with systems that fol¬ 
low a continuous time Markovian process in the contin¬ 
uous state space, the phase space. We consider a system 
of particles that follows a dynamics described by the fol¬ 
lowing set of Langevin equations, interpreted according 
to ltd, 

di) ■ 

= Fi{x)-aiV^+Si{t), (145) 

at 

where m is the mass of each particle, Vi = dxi/dt and 
Xi is the position of the z-th particle and x denotes the 
vector X = (xg,..., Xn)- We will also use the notation 
V = (vi,... ,Vn)- The quantity Fi{x) is the force acting 
on the z-th particle, and S'i(t) is a stochastic variable with 
the properties 


^ W:^P^{a,k - a,k) + ^ W:^P^{x^, - x]) = 0, (140) 

3 3 

valid for r + 1 < A: < 

^ = 0’ (141) 

j 3 

valid for 1 < £ < r. These last two equations determine 
ajfc and R^g. 

Let us consider now the expansion of the rate of the 
extent of reaction xi, given by (11091) . Replacing the ex¬ 
pansion (I134D into (I109L we get an expression linear in 
Am and A^fc. The coefficient of Am is the Onsager coef¬ 
ficient Lgm which is given by 

Lgm = Y. - ^j)- ( 142 ) 

ij 

Next we use equation (I141|) to write the Onsager coeffi¬ 
cient in a more appropriate form. To this end we proceed 
as follows. We multiply (11411) by cr)" and sum in z to get 
a first equation. Next, we multiply (11411) by and sum 
in z to get a second equation. From these two equations 
we get an equation for the right-hand side of (|142l) from 
which we reach the following expression 

Lgm = ^Y 

^3 


- ^ E W:,P^{R.g - Rjg){R,m - Rj^). (143) 


From this expression it follows that 


(5z(0)=0, (146) 


(l?z(i)5,(i')) (147) 


where Bij may depend on x and v. 

Notice that we are considering the so called under¬ 
damped systems, for which the state of a particle is de¬ 
fined by its position and velocity [s^, in opposition to 
the overdamped case, for which the state of a particle is 
defined only by its position [^ . 

The quantities ^i{t) are random forces acting on the 
particles including the ones that describe the contact of 
the system with the environment. We will treat two 
cases: one in which the system is isolated (microcanonical 
ensemble) and the other in which the system is in con¬ 
tact with a heat reservoir (canonical ensemble). In the 
first case the forces Fi are conservative and the stochastic 
forces are set up in such a way that the energy is con¬ 
served in any stochastic trajectory. In thermodynamic 
equilibrium they will lead to the Gibbs microcanonical 
probability distribution. In the second case the forces F) 
are also conservative and the random forces are set up in 
such a way that in thermodynamic equilibrium they will 
lead to the Gibbs canonical distribution. 

Using the Ito interpretation, we can show that the 
Langevin equations (11451) are associated to the following 
Fokker-Planck equation 


dP 




E oti d , \ 1 v-^ 




dvidv 


■{B,,P\ (148) 


— Blm: (444) 

which is the Onsager reciprocal relation . 


equation that gives the time evolution of the probability 
distribution P{x,v,t) of x and v at time t. It is conve¬ 
nient to write down the Fokker-Planck equation in the 
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following form 


dP 


dP 




dvi 


where Ki and Ji are given by 

dP F, dP 


K,, = ' 


* dxi m dvi ’ 


and 


p = -^v,P 

yjl qryf ^ f J , . 


m? ^ dv-i 


(149) 


(150) 


(151) 


Let us consider now the time variation of entropy S', 
given by 

S{t) = —ks j P{x,v,t)\nP{x,v,t)dxdv. (152) 


where ^ij are stochastic variables with the antisymmet¬ 
ric property = —^ij- The multiplicative noise at 
the right-hand side changes the velocities of the parti¬ 
cles while keeping the kinetic energy invariant and can 
be interpreted as random elastic collisions of the parti¬ 
cles with themselves or with immobile scatters. A similar 
noise has been used do describe a particle that moves at 
constant speed but changes direction at random times 


Multiplying (11571) by Vi and summing in i we may con¬ 
clude, after using the antisymmetric relation 
that E{v,x) is strictly conserved along any stochastic 
path x{t), v{t). Therefore, the equation of motion (11571) 
describes a system of particles evolving in time in such 
a way that the energy is strictly constant. In analogy 
with equilibrium statistical mechanics, this defines a mi- 
crocanonical ensemble. 

The stochastic variables (f) are defined by the rela¬ 
tions 


The derivative of S gives 


(e.,(t)) = 0, (158) 


dt 


= -ks 



In Pdxdv. 


(153) 


After replacing (11491) into this equation and performing 
appropriate integrations by parts we reach the following 
expression 


dt 



dxdv. 


(154) 


The terms corresponding to Ki vanish, that is. 


and 


(159) 


where pj > 0 is a parameter that gives the strength of 
the stochastic noise. Using the Stratonovich interpreta¬ 
tion, and taking into account the antisymmetric property 
^ji = —^ij of the noise, we may write down the associate 
Fokker-Planck equation, given by 


dP 




m 


dv^ 


— fcs J Kiln Pdxdv = 0. (155) 

i 

We are assuming that P and its derivatives vanish at the 
boundary of integration. 


B. Microcanonical ensemble 


Here we treat the case of an isolated system, with 
no contact with the environment so that the energy is 
strictly conserved. We thus assume that the force Fi are 
conservative so that A) = —dV/dxi which allows us to 
define the energy function E(y,x) as 

E{x,v) + V{x). (156) 


The strict conservation of energy means to say that 
E{x,v) should be a constant along any stochastic tra¬ 
jectory in phase space. This condition is fullfiled by the 
following set of Langevin equations, understood accord¬ 
ing the Stratonovich interpretation. 



Fi{x) -|- ^ ^ 



' d dP d dP\ 


(160) 


equation that gives the time evolution of the probability 
distribution P{x,v,t) of x and v at time t. The last 
summation extends over i p j and we recall that Xji = 
> 0 . 

It is worth mentioning that equation (11571) . understood 
in the Stratonovich sense, is equivalent to the following 
equation, interpreted according to Ito interpretation. 


in- 


djM 

dt 


= Fi{x) - UiV, + ^ ^ijVj, 


(161) 


where 


Q!i = ^ Xij. (162) 


Of course, this equation leads to the same Fokker-Planck 
equation (11601) . 

It is convenient to write down the Fokker-Planck equa¬ 
tion in the form given by (11491) where Ki is given by (11501) 
and Ji is given by 


Ji - ^ ( dj 




PJ'i-) 


(157) 


(163) 
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1 f dP dP\ 

Let us determine now the time derivative of entropy, 
which is given by equation p54l) . After replacing (116311 
into equation (11541) and performing appropriate integra¬ 
tion by parts we reach the following expression 


case in which each particle i is in contact with a reser¬ 
voir at temperature Ti. The appropriate set of Langevin 
equations that describes this situation is given by 


= Fi{x) - + (i(t), (171) 

at 

where Ci (0 ^ stochastic variable with the properties 
m)) = 0, (172) 


We are assuming that P and its derivatives vanish at 
the boundary of integration. The right-hand side of this 
equation is clearly nonnegative and is therefore identified 
as the entropy production rate 


n = 

i<j 


1 

P 


( dP dP\ 


2 

dxdv, 


(166) 


which can also be written in the form 


n = 



(167) 


where the summation is over ij such that A^- 7 ^ 0 , so that 



(168) 


In the present case there is no entropy flux, 


$ = 0 , 


(169) 


which is consistent with our interpretation that equations 
(11571) describe an isolated system. Taking into account 
that n > 0 it follows at once that dS/dt > 0 for an 
isolated system. 

In the stationary state, which is a thermodynamic 
equilibrium, the probability distribution P®(a;,u) de¬ 
pends on X and v only through E{x, u), that is, P'^{x, v) is 
a function of E{x,v). This statement can be checked by 
substitution on the right-hand side of the Fokker-Planck 
equation (11601) . Since E{x, y) is invariant along any path 
in phase space and supposing that initially its value is U, 
it follows that 

P%x,v) = ^iU - E{x,v)), (170) 

where is a normalization constant that depends on U. 
We remark that in this case 11, given by (11661) . vanishes, 
as expected. 


C. Canonical ensemble 

Now we consider the case of a system in contact with a 
heat reservoir. In fact, we will consider the more general 


{QitKjit')) = 2a,kBT, 6,,Sit - t'), (173) 


where Ti and ai are parameters. The two last terms in 
equation (11711) are interpreted as describing the contact 
of the j-th particle with the heat bath at a temperature 
Ti and ai is the strength of the interaction with the heat 
reservoir. 

To the set of Langevin equations (I17ip is associated 
the Fokker-Planck equation 


dP 



OiiTi—iViP) 

OV,, 


ks 

w? 


y^^ajT^ 


d^P 


(174) 


equation that gives the time evolution of the probability 
distribution Pix, v, t) of x and v at time t. 

The Fokker-Planck equation can again be written in 
the form given by (11491) where Ki is given by p50l) and 
Ji is given by 


ctiVt p _ aiksTj dP 
m m? dvi 


(175) 


Again the derivative of entropy is given by (IT541) . Re- 
|?lacing (11751) into (11541) we get the following expression 


dt 


V [ 

\a,E P 


m 

—1 

T 


ViJi I dxdv. 


(176) 


The summation in (IT7S1) extends only to the terms for 
which ai ^ Q and Ti ^ 0. 

The first term on the right-hand side of equation (11761) 
is nonnegative and is identified as the entropy production 
rate [s^ 


^ a^Ti J P 


(177) 


Although this identification may seem to be arbitrary, 
as has been argued we will see in the next section 
that in fact it is in accordance with the expression ([H]) . 
It vanishes only when Ji = 0 which is the equilibrium 
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condition. The second summation is thus the entropy 
flux 

$ = / v^Jidxdv, (178) 

* J 

which can also be written as 



After replacing given by (jl75L into (11781) and per¬ 
forming an integration by parts, the variation of the en¬ 
tropy of the system becomes 

f = n-$. (180) 

Let us assume that the forces are conservative, Fi = 
—dVjdxi- In this case, we define the energy of the system 
as 

E{x,v) = '^^vl+ V{x). (181) 

i 

Using the Fokker-Planck equation in the form (11491) we 
get the following expression for the time derivative of the 
average energy U = {E(x,v)) 


where Z is a. normalization constant. 

If we integrate equation (II85I) in time, from an initial 
time to until infinity, when the system is in equilibrium, 
we get 

S-So-^{U-Uo)>0. (187) 

Let us suppose that the system is in contact with just 
one heat reservoir at temperature T and that is tempera¬ 
ture is varying slowly so that dT/dt = a is small. This is 
again the quasi-static process that we have already dis¬ 
cussed. In this case, the quantity Ji will be of the order 
a so that 11 will be of the order . On the other hand, $ 
remains at the linear order in a and we may write from 
(11851) dS/dt = {l/T)dU/dt. It follows that the entropy 
and energy cannot be arbitrary but are connected by the 
relation TdS = dU so that a system performing a quasi¬ 
static process may be considered to be in equilibrium. 
From the result (I187p we see that the curve that connect 
U and S has the property of convexity. To perceive this 
it suffices to imagine that at the initial time to the energy 
Uo and entropy So correspond to values of equilibrium at 
a certain temperature To- 

D. Nonequilibrium stationary state 


where 


dt 


= 


(182) Let us take a look at the the energy variation per unit 
time, or power, Vi associated to the z-th particle, given 
by 


$„ = —/ ViJidxdv (183) 

i 

is the flux of energy from the system to outside. To 
reach this expression we have performed appropriate in¬ 
tegration by parts and assumed that P and its derivatives 
vanish at the boundaries of integration. Using the defini¬ 
tion of Ji, given by (HZSl), we may write the energy flux 
as 

$„ = y] a, . (184) 

When all temperatures are the same V = T we have 
$ = ^u/T so that 


dS 1 dU 
dt T dt 


(185) 


From which follows that the time variation of F = U—TS 
is given by dF/dt = —Til so that dF/dt < 0. 

Thermodynamic equilibrium occurs when all temper¬ 
atures are the same, Ti = T, and the forces are conser¬ 
vative, Fi = —dVjdxi. In this case, Ki — Q and Ji = 0, 
which leads to the following result for the equilibrium 
probability distribution 


P^{x, v) 


}_-E{x,v)/kBT 

z 


Fi = (u.Fi)-h y^(z;,2), (188) 

where the first term is associated to the dissipation due 
to the force Fi acting on the particle and the second the 
time variation of its kinetic energy. Using the Fokker- 
Planck equation it is straightforward to shown that 

Vi = ar{v1) - (189) 

m 

Therefore, 


= (190) 

i 

and 

i 

I 

and we recall that the summation is over i such that 
tti ^ 0 and Ti ^ 0. 

Let us consider the contact of the system with two 
reservoirs A and B at temperatures Ti and T 2 , respec¬ 
tively. The heat flux from reservoir A to the system is 
given by 


i^A 


(186) 


( 192 ) 
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where the summation is over the particles that are in 
contact with the reservoir A. A similar expression holds 

ieB 

for the heat flux J' from reservoir B to the system. In 
the stationary state, 11 = $, and taking into account the 
expression (119111 for <&, we get 

n = f + ^. (194) 

But in the stationary state 4>„ = = 0 so that 

n = XJ, (195) 

where 

X = (196) 

Let us assume that X is small so that AT = T 2 — Ti 
is small. In this case J = LAT where L is the ther¬ 
mal coefficient. Writing the probability distribution as 
P{x,v) = P^[x,v)[l — ATa{x,v)], where P‘^{x,v) is the 
equilibrium distribution when the temperatures of the 
reservoir is Ti, we may calculate J to get 

L = ^^ai / vfa(x,v)P'^{x,v)dxdv, (197) 

i&A 

which may be undertood as an average over the equilib¬ 
rium distribution. 


V. MASTER EQUATION REPRESENTATION 
OF THE FOKKER-PLANCK EQUATION 

A. Microcanonical ensemble 

It is possible to represent the Fokker-Planck in terms 
of a master equation. This can be done by a discretiza¬ 
tion of the phase space in a such a way that the contin¬ 
uum limit will reduce the master equation to the Fokker- 
Planck equation. From the representation we can easily 
identify the transition probabilities from which we can 
obtain, for instance, the entropy production rate. 

To set up the discrete stochastic dynamics we imag¬ 
ine a representative point in the phase space following 
a stochastic trajectory. We consider two types of transi¬ 
tions from a given point in the phase space. The first type 
is defined by the transitions determined by the Hamilto¬ 
nian flow. This type of transition is defined by 

{x,v)^{H+x,H+v), (198) 

where x and v are vectors with the same compo¬ 
nents of the vectors x and v except the i-th components 


Xi and Vi which are transformed to x[ and u', where x' 
is given by 

x[= Xi + bvi, (199) 

and is determined in such a way that the energy is 
conserved, that is, 

E{H+x, H+v) = E{x, v), (200) 

where 6 > 0 is a parameter. Each transition occurs with 
rate 1/6. If 6 is sufficient small, u' is given by 

Vi = Vi + b—. (201) 

m 

Notice that the Hamiltonian transition defined above has 
no reverse in the sense that from a point x, v) we 
cannot reach the point (x, v) with this type of transition. 

The transitions of the second type changes only the 
velocities and preserves the kinetic energy. This type is 
defined by 

(x,u) (x,Myx), ( 202 ) 

where MijV is a vector with the same components of the 
vector V except the components i and j which are v[ and 
Vj given by 

vl = ViCOsO — VjSinO, Vj = ViSinO + vj cos6, (203) 

where 0 > 0, so that (u')^ -I- (u')^ = vf + Vj and the 
kinetic energy is preserved. Another possible transition 
is defined by 

(x, t>) —>■ (x, Mjiu). (204) 

Each of these transition occurs with rate equal to 
Xijlm?9'^. Notice that this second type of transition 
has a reverse since from the point {x,Mijv) it is pos¬ 
sible to reach the point (x,v). It suffice to observe that 

Mji{Mijv) = V. 

The transitions above lead us to the following master 
equation 

^P(x, u) = ^ 

i 

+ ^ Mijv) + P{x, Mjiv) - 2P{x, u)}, (205) 

u' 

where H~ is defined in a way similar do except that 
the sign in front of 6 in equation (I199p is negative. It is 
straightforward to show that expression (I205|) reduces to 
the Fokker-Planck (11601) in the limit e —5> 0 and 6 —>■ 0. 

Taking into account that all transitions preserve the 
energy E{x, v) we see that in equilibrium the probability 
distribution P'^{x,v) depends on (x,u) through E(x,v). 
If at initial time the energy is equal to t/, then P®(x,u) 
vanishes if P(x, v) ^ U and is a constant if E(x, v) = P, 
which is the Gibbs microcanonical distribution. 
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B. Canonical ensemble 


C. Entropy production 


Next we set up a discrete stochastic dynamics, de¬ 
scribed by a master equation, whose continuous limit 
gives the Fokker-Planck equation (jl74l) . The represen¬ 
tative point in phase space {x, v) performs a stochastic 
trajectory. We consider again two types of transitions 
from a given point in the phase space. The first type is 
the transition defined by the Hamiltonian flow given by 
equation (11981) . The transition of the second type changes 
only the velocities but in general it does not preserve the 
kinetic energy. This type of transition is defined by 

(x,v)->■ (x,C^^v), (206) 

where C^v is a vector with the same components of the 
vector V except the Tth component v' which is given by 


We have seen that the entropy production rate of a 
system described by a master equation is obtained by 
expression This expression is appropriate when the 
rates of the reversed transitions are nonzero. This is the 
case of transitions defined by (12021) . The entropy produc¬ 
tion rate Hm associated to these transitions, according 
to ([3]) is given by 

Hm = -y ^ ^ - P(x, u)} x 

x,v ij 


P{x,Mijv) 
X In — 

P{x, v) 


(213) 


v'i = Vi± a, (207) 

where a > 0 is a parameter and each transition occurs 
with rate 

A^{v) = _ (2O8) 

The master equation is written as 

^ 1 
— P(a;, v)=J2 t 


= ^{H+(Q v)P{x,C, v)-A^ {v)P{x,v)} 

i 


+ J2{A-iC+v)Pix,C+v) - Ativ)Pix,v)}. (209) 

i 

It is straighfoward to show that in the limit a —>■ 0 and 
6 —5> 0, the master equation reduces to equation (I174L 
and the master equation can indeed be understood as a 
representation of the Fokker-Planck equation (I174p . 

The stationary solution of the master equation when 
all temperatures are the same, which corresponds to the 
thermodynamic equilibrium, is the Gibbs distribution. 
Indeed, the detailed balance of the master equation gives 
us the relation 


P^(ff-x,ff-v) = P^(x,v), (210) 

which means that P'^(x,v) depends on (x,v) through 

E{x,v). Writing 

P(a;,z;) = (211) 

Zi 

we see that the other relation, 

At(C-v)P^(x,C-v) = A-(v)P^(x,v), (212) 

is fullfiled if we take into account that all temperatures 
are the same, Ti = T. 


In the limit 0 —> 0, the right-hand side reduces to ex¬ 
pression on the right-hand side of (II65I) . The entropy 
flux associated to the transitions (12021) is obtained 
by using dUD, but it vanishes identically, 

$M = 0. (214) 

Let us consider now the entropy production rate He 
associated to the transitions defined by (12061) . According 
to expression ©, it is given by 

nc = ^^^{A+(C-u)P(a;,C'-u)-A-(u)P(a:,u)}x 

x,v i 


.,^ At(C-v)P(x,C-v) 

A-(v)P(x,v) 


(215) 


After taking the limit a —>■ 0 this expression is reduced 
to the result (11771) . The corresponding entropy flux $<7 
is obtained from m and is given by 


v)P{x,C^ v)-Ai {v)P(x,v)}x 

x,v i 


X In 


AtiC-v) 

A~ (v) 


(216) 


The limit a —>■ 0 leads us to the result (|178l) . 

We now wish to consider the entropy production rate 
and the flux of entropy coming from the parts of the 
stochastic trajectory associated to the Hamiltonian flow, 
given by the transitions defined by (I200|) . We postulate 
that the entropy flux associated to the Hamiltonion flow 
vanishes identically, — 0. Therefore, the entropy pro¬ 
duction rate associated to the Hamiltonian flow should be 
equal to part of dS/dt coming from the Hamiltonian flow. 
This part can be obtained by inserting the first summa¬ 
tion of the right-hand side of the master equation (12051) 
into equation (|5]). After doing this, we get the following 
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expression for the entropy production rate associated to 
the Hamiltonian flow, 

Hh ^ fa i: i: .) In , (217) 

x,v i ^ 

which is similar to expression 

Next we have to show that Hjj > 0. To this end, we 
expand each term in the summation in powers of b. Up to 
linear terms in b the i element of the summation equals 



But the integral in x and v of the first term vanishes 
so that > 0. In fact, it vanishes in the continuum 
limit 6 —>■ 0. Therefore, in the continuum limit H^f = 0. 
From this result it follows that Hm is the total produc¬ 
tion of entropy for the microcanonical case. Since in the 
continuous limit, it goes into (11671) . it follows that the ex¬ 
pression given by (11671) is indeed the entropy production 
rate, as we have assumed. Similarly, it follows that He 
is the total production rate for the canonical case. In the 
continuous limit it is identified with (I177p so that the ex¬ 
pression given by (11771) is indeed the entropy production 
rate, as assumed. 

VI. CONCLUSION 

We developed the stochastic approach to thermody¬ 
namics based on the stochastic dynamics. More specifi¬ 
cally, we used the master equation, in the case of discrete 
state space, and the Fokker-Planck, in the case of con¬ 
tinuous state space. Our approach is founded on the use 
of a form for the production of entropy which is non¬ 
negative by definition and vanish in equilibrium. Based 
on these assumptions we studied interacting systems with 
many degrees of freedom in equilibrium or out of thermo¬ 
dynamic equilibrium and how the macroscopic laws can 
be derived from the stochastic dynamics. This required 
the introduction of the transition rates which thus play a 


fundamental role in the present approach, similar to the 
Gibbs distribution in the case of equilibrium. 

Using the property that the production of entropy is 
nonnegative, which is understood as the dynamic formu¬ 
lation of the second law of thermodynamics, we were able 
to show that in the quasi-static process, the representa¬ 
tive point in the thermodynamic space approaches a sur¬ 
face and that this surface has the property of convexity. 
These statements are usually introduced as postulates in 
equilibrium thermodynamics. We have also shown the 
bilinear form of entropy production, which is a sum of 
terms, each one being a product of a force and a flux. We 
remark that this is the macroscopic form used in nonequi¬ 
librium thermodynamics, and should not be confused 
with the microscopic definition of entropy production it¬ 
self, which looks like a bilinear form. From the bilinear 
form of the entropy production, we have determined the 
Onsager coefficients and shown that they obey the re¬ 
ciprocal relations. The nonequilibrium steady states of 
a system with several chemical species and chemical re¬ 
actions were studied by the use of appropriate transition 
rates. From the dehnition of the entropy production rate 
it was possible to derive the bilinear form, which in this 
case is written in terms of affinities and the rates of the 
extents of reaction. In equilibrium the affinities vanish, 
which is the condition for chemical equilibrium. 

Using appropriate transition rates or appropriate 
stochastic noise, in the case of the Fokker-Planck, it was 
possible to study several situations that were analogous 
to those related to the microcanonical, canonical and 
grand canonical Gibbs ensembles. For the microcanon¬ 
ical case in continuous state space we have introduced 
an energy conserving stochastic noise. For the canonical 
case we used the usual white Gaussian noise. To make 
contact with the master equation, we have used a master 
equation representation of the Fokker-Planck. Using this 
representation we confirmed the expression for the pro¬ 
duction of entropy that was introduced by the splitting of 
the time derivative of entropy. In this case we postulated 
that a Hamiltonian transition induces no flux of entropy. 
Since the entropy is constant along a Hamiltonian flow in 
continuous space, this implies no production of entropy. 
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